Talking to your computer!

Anyone can talk to their computer! However, it just sits there like a rock not knowing you even said anything. If it does, then it sure as heck wouldn't understand you!

Microsoft has solved this problem. With one simple ActiveX control, we have the ability to talk to our computer. And with another simple control, we can add a professional character too!

• First, we need to download the Speech Recognition Engine (6 MB)
• Download and install it using the link above
• Click HERE for the download site. Click on the Speech Control Panel button and download it.
• After it is installed, click on the first link in the index. Download and install it.
• Click on the link below it, and choose a character from the dropdown list. I like Robby the robot the most. There is also a bird (Peddy), and Genie, and Merlin. You can use the others later. If you do not choose Robby, things will not make sense.
• Then click on the download button to download the character. Install it, then continue with the next step.
• Now download the next two links, and install those. That will allow the computer to speak back to us. You should have clicked on every link but one.

So far, that should be easy. If you could not follow it, we clicked on every link in the contents at the top of the page except the bottom one. It may seem like we are ready, but we really aren't...

If you try to speak to the computer now, it will have a very hard time "understanding" you. So, we need to train it.

• Enter the Control Panel. It can be found in the Start Menu.
• Find the Speech icon, and click on it. A dialog will open.
• Click on the "Other" tab at the top, the second one.
• Click on the "SAPI 4 Control Panel..." button to open yet another dialog.
• Click on the text in the list that reads "Microsoft Speech Recognition Engine 4.0 (English)" at the bottom.
• Click on the "Training..." button. Another dialog will open.
• NOTE: You may have done this already during the setup. It is wise to do it again, though.
• Choose a session from the list. None are too exciting...
• Click on the 'Next >' button to take you to the next screen.
• Get your microphone ready, because once you click on 'Next >' again, you will have to speak!
• It will ask you to read the story. Do NOT read abnormally! Read like you normally do so it learns how you speak.
• When you are done, it will update the computer. Keep doing it until you seem comfortable. The more times you do it, the better it will get to know you!

Once that is done, you computer will better understand you. It takes a lot of work to get it good. Mine gets it correct about 90% of the time, but I still need work.

Now we are finally ready to start making our program. As I sit here typing this, I wonder exactly what this will do. Well, I now know, and you will know soon enough. Just follow all of the steps to get the final working program and be amazed at what Microsoft can do for us...

• Open Visual Basic and start a new Standard EXE project.
• Rename the form 'frmMain' and delete its caption property. The form will never be visible, so we do not need to worry about this.
• Set its Visible property to 'False' so we can not see it. We only want to see our little Agent.
• We now need to add the ActiveX controls to our project.
• Right-click on the toolbox and select 'Components'
• In the list, find and select Microsoft Agent Control 2.0 and Microsoft Direct Speech Recognition
• You will see an ear and an agent appear in your toolbox.

We now have everything we need added to our project. We can now start to make it happen! We now must create a list of our commands to be loaded from a text file.

Grammar

Open up Notepad. We must create our own file to load the grammar and vocabulary from. It will only recognize what we enter in this document. So, enter the following:

[Grammer]
type=cfg
[<start>]
<start>=Show Yourself

<start>=Hide Yourself

<start>=Speak

<start>=Execute Notepad

<start>=Execute Microsoft Paint

<start>=Execute Microsoft Visual Basic

<start>=Display Character Map

<start>=Disallow Commands

<start>=Reallow Commands

<start>=Hibernate

<start>=Make Repairs

<start>=Tell Me a Joke

<start>=What Is My Fortune

<start>=Play Number Guessing Game

<start>=Status Report

<start>=0

<start>=1

<start>=2

<start>=3

<start>=4

<start>=5

<start>=6

<start>=7

<start>=8

<start>=9

<start>=10

<start>=Animate

<start>=Display Animation List

<start>=Display System Information

<start>=System Shutdown

<start>=System Reboot

Save it in the same directory as the project, naming it 'words.txt'. Now that our grammar list has been created, we can load it into our program. We first need to place the ActiveX controls onto our form.

ActiveX Controls

Drag the two new controls onto the form (the ones that look like an agent and an ear). Name the ear-like one 'Reco', for it will be our Speech Recognition Engine. Then, set its 'Visible' property to 'False'. Next, name the agent 'Agent', so all you have to do is erase the one. This does not have a visible property.

Our Agent

Before we load the list we just created, let's create our Agent object. You do not have to do it in this order, but I just want you to see what our character will look like. Create a new module and name it 'modMain'. Enter the following code in the General Declarations section (at the top):

Public Bot As IAgentCtlCharacterEx

This creates a new Agent character from an external source. Therefore, we will have to load it. Double click on frmMain and enter the following code:

Agent.Characters.Load "Robby", "robby.acs"
Set Bot = Agent.Characters("Robby")
Bot.Show

Bot.Play "Wave"
Bot.Speak "Hello!"
Bot.Speak "How are you doing today?"

Bot.MoveTo 200, 200

That is it! If you run the program now, you will see your robot wave to you, speak, then move. Let's take a look at the code.

The first two lines load the bot. The 'robby.acs' is the file we load it from. The third line shows our bot. The Bot.Play "Wave" line will play his 'Wave' animation. We will take a look at animating much more in the future. Then, we tell the bot to speak with Bot.Speak. Finally, the MoveTo command will move the bot to a different part of the screen.

Before you move on, try to do some different things with it. Modify what it says when it appears, move it to a different position on the screen.

Loading the Grammer

We can not speak to the computer until we load the list we just created. Enter the following code above all the code we just put in the Form_Load procedure:

Reco.GrammarFromFile App.Path & "\Words.txt"
Reco.Activate

To see if it loaded correctly, we will make a simple example. If you speak a phrase we created, it will output it to the debug window. Please note that it may not recognize you every time. Only speak what is in our list. Double-click on Reco and enter this in its Phrase_Finish procedure:

Debug.Print Phrase

As you speak, look in the Debug Window. If it outputs any of the phrases listed in Words.txt, we can continue.

Responding

How do we respond to this? Well, we need to replace the one line of code we just entered with a little more:

On Local Error Resume Next
Debug.Print Phrase

Select Case Phrase
    Case "Show Yourself"
        Bot.Show

    Case "Hide Yourself"
        Bot.Hide

End Select

Please note that it is case sensitive, so you must type it exactly as it appears above. All we do is output the phrase said to the Debug Window, then check it. These two phrases are the basic ones, they show and hide the bot. If it is 'Show Yourself', then the bot will be displayed. If it is 'Hide Yourself', we will hide the bot. It is that simple. Try it out and see if it works for you. Please be patient, it may take a while to recognize...

Speaking

We will now program our bot to speak to us. Please modify the above code so it looks like this:

On Local Error Resume Next
Debug.Print Phrase

Select Case Phrase
    Case "Show Yourself"
        Bot.Show

    Case "Hide Yourself"
        Bot.Hide
 

    Case "Speak"

        BotSpeak

End Select

In the above code, we only added two lines. It will call a sub procedure which will randomly choose something for our bot to say. In modMain, create a new sub procedure named 'BotSpeak':

Public Sub BotSpeak()

    Randomize

    Dim r As Integer
    r = Int(16 * Rnd)

    Select Case r
        Case 0
            Bot.Speak "I love you!"

        Case 1
            Bot.Speak "You are amazing!"

        Case 2
            Bot.Speak "Make me a sandwich!"

        Case 3
            Bot.Speak "I'm tired..."

        Case 4
            Bot.Speak "You little brat!"

        Case 5
            Bot.Speak "Dreams do come true!"

        Case 6
            Bot.Speak "Solve the equation 8(2x + 3) - (5x + 3x + 4)"

        Case 7
            Bot.Speak "What is the meaning of life?"

        Case 8
            Bot.Speak "I hate you, go away!"

        Case 9
            Bot.Speak "Make me!"

        Case 10
            Bot.Speak "I'm so happy being with you!"

        Case 11
            Bot.Speak "Have you read the book 844454XC50?"

        Case 12
            Bot.Speak "I want some self time..."

        Case 13
            Bot.Speak "Humans are obsolete!"

        Case 14
            Bot.Speak "I can not wait to take over your world!"

        Case 15
            Bot.Speak "South Park rules!"

    End Select
End Sub

You should be able to understand that. We first choose a random number. Then, we tell the bot to speak the text accociated to that number. That is it! Run the program, say "Speak", and see what it says. Feel free to change the text to whatever you'd like.

Executing Applications

This is a very easy topic to cover. All you have to do is add the following code to what we currently have in the Phrase_Finish procedure:

Case "Execute Notepad"
    Shell "C:\Windows\NOTEPAD.exe", vbNormalFocus

Case "Execute Microsoft Paint"
    Shell "C:\WINDOWS\SYSTEM32\mspaint.exe", vbNormalFocus

Case "Execute Microsoft Visual Basic"
    Shell "C:\Program Files\Microsoft Visual Studio\VB98\VB6.exe", vbNormalFocus

Case "Display Character Map"
    Shell "C:\WINDOWS\SYSTEM32\charmap.exe", vbNormalFocus

The 'Shell' command will execute an EXE file. The first parmeter is the Path, where the executable is located. The second is the focus. Do you want it to have the focus, do you want it to be minimized, maximized, or normal?

Disallow and Reallow

Before we get started, add a public Boolean variable named 'Disallow'. Now, add the following code at the top of the Phrase_Finish procedure:

If Phrase = "Reallow Commands" And Disallow Then
    Bot.Play "DoMagic1"
    Bot.Play "DoMagic2"

    Bot.Play "Idle1_1"
    Disallow = False
    Bot.Speak "Welcome back!"
End If

If Phrase = "Disallow Commands" Then
    Bot.Play "DoMagic1"
    Bot.Play "DoMagic2"

    Disallow = True
    Bot.Play "Idle1_1"
    Bot.Speak "I will wait for your return!"
    Exit Sub
End If

If Disallow Then Exit Sub

Run the program and say 'Disallow Commands'. Now, try to get it to speak. You can't. Now say 'Reallow Commands'. It can speak again! This is good if you don't want someone messing around with your bot.

Hibernating and Making Repairs

These are only two animations. Hibernating will, of course, but it to sleep. Make Repairs will make him take off his head, and make repairs. Add the following code:

Case "Make Repairs"
    Bot.Play "Idle2_2"

Case "Hibernate"
    Bot.Play "Idle3_2"

You should understand that tiny amount of code. These two animations can also be played if you leave your bot alone long enough.

Jokes

This is the same concept as speaking. You can just copy the BotSpeak procedure and replace the lines with jokes. I will give you two. You can come up with some of your own. Enter the following code in the Phrase_Finish procedure:

Case "Tell Me a Joke"
    TellJoke

Now, create the sub procedure in modMain:

Public Sub TellJoke()
    Dim r As Integer
    r = Int(2 * Rnd)

    Bot.Play "GetAttention"

    Select Case r
        Case 0
            Bot.Speak "Why didn't the chicken cross the road?"
            Bot.Speak "Because he was chicken!"

        Case 1
            Bot.Speak "Why didn't the man cross the road?"
            Bot.Speak "Because he didn't want to!"
            Bot.Play "Confused"

    End Select

    Bot.Play "GetAttentionReturn"
End Sub

Say "Tell Me a Joke" and listen. If you have any good jokes that are yours (I have millions of them, but they are not mine. That is why they are not on here), feel free to send them to me!

Fortune Telling

Again, this is very similar to joke telling and speaking. Enter the following lines in the Phrase_Finish procedure:

Case "What Is My Fortune"
    TellFortune

Now, create the procedure in modMain:

Public Sub TellFortune()
    Dim r As Integer
    r = Int(14 * Rnd)

    Bot.Play "Process"

    Select Case r
        Case 0
            Bot.Speak "Do not go outside tomorrow..."

        Case 1
            Bot.Speak "Keep looking behind your back!"

        Case 2
            Bot.Speak "In the sky is your answer!"

        Case 3
            Bot.Speak "ERROR - DEATH TOO SOON!"

        Case 4
            Bot.Speak "Hold on to your belongings..."

        Case 5
            Bot.Speak "I see great things in your future!"

        Case 6
            Bot.Speak "Do NOT answer the door!"

        Case 7
            Bot.Speak "tomorrow will be a great day for you!"

        Case 8
            Bot.Speak "It says - 'NEXT BIRTHDAY'"

        Case 9
            Bot.Speak "I'd rather not say!"
            Bot.MoveTo 10, 10

        Case 10
            Bot.Speak "Take shelter!"
            Bot.Hide

        Case 11
            Bot.Speak "You will be married tomorrow!"

        Case 12
            Bot.Speak "You have a great day ahead of you!"
            Bot.Think "I lied"

        Case 13
            Bot.Speak "ERROR - MISPRINT"

    End Select
End Sub

There is something new up there. Bot.Think. Instead of speaking, the agent displays a cloud with the text in it. The rest we have already gone through.

Number Guessing Game

This is a tiny bit more involved. First create a Boolean variable in modMain called InGame, and an Integer variable named GuessNum. Now, put this at the top of the Phrase_Finish sub:

If InGame Then
    GameInput Phrase
    Exit Sub
End If

Now, add this to the same procedure:

Case "Play Number Guessing Game"
    PlayGame

We have two procedures to create. Let's start with PlayGame:

Public Sub PlayGame()
    GuessNum = Int(11 * Rnd)

    Bot.Play "Explain"
    Bot.Speak "You have three tries to guess the number I am thinking of."
    Bot.Speak "I will tell you if you are too high or too low..."

    Bot.Play "Process"
    Bot.Speak "I am thinking of a number between 0 and 10..."

    InGame = True
End Sub

This procedure explains what you are to do, and chooses a number. It then sets the InGame variable to 'True' allowing us to bypass all other commands and worry only about the numbers we are guessing:

Public Sub GameInput(Phrase As String)
    If Phrase = "0" Or Phrase = "1" Or Phrase = "2" Or Phrase = "3" Or Phrase = "4" Or Phrase = "5" Or Phrase = "6" Or Phrase = "7" Or Phrase = "8" Or Phrase = "9" Or Phrase = "10" Then

        Dim tmpNum As Integer
        Static GNum As Integer

        If GNum = 3 Then GNum = 0
        tmpNum = Int(Phrase)

        If tmpNum > GuessNum Then
            Bot.Speak CStr(tmpNum) & " - Number too high"

        ElseIf tmpNum < GuessNum Then
            Bot.Speak CStr(tmpNum) & " - Number too low"

        ElseIf tmpNum = GuessNum Then
            Bot.Play "Congratulate"
            Bot.Speak CStr(tmpNum) & " - That is correct!"
            If GNum = 0 Then Bot.Speak "WOW! First Try!"

            InGame = False
            Exit Sub
        End If

        GNum = GNum + 1

        If GNum = 3 Then
            Bot.Play "Sad"
            Bot.Speak "You did not get it..."
            Bot.Speak "The correct number was " & CStr(GuessNum) & "..."

            InGame = False
        End If
    End If
End Sub

This is really some simple code. I simplified the first line so it is easier to understand. We check to see if what the user said is actually a number. If it is, we continue. If not, we bypass it. Then, we reset the tries count (GNum). Then, we change the string into an integer and store it in tmpNum. The bot will then inform us if it is too high or too low. If we are correct or take our three turns, the game is over.

Status Report

This is nothing good. I just added this because every robot must have status reports. It is also here if you decide to turn your robot into a pet like a tamagotchi (I did). Every time, the bot will say he is operating a optimum efficiency:

Case "Status Report"
    Bot.Play "Explain"
    Bot.Speak "I am operating at optimum efficiency!"

Animation

If you've looked at Microsoft Word or any of the Office products, you can animate the agents. Yes, they are agents, so keep this in mind: YOU CAN USE THEM IN YOUR APPS!

First, we are going to create the Animation List. Follow these simple steps:

• Create a new form and name it frmAni

• Add a ListBox control to it, naming it lstAni

• Add three CommandButtons: cmdPlay, cmdStop, and cmdCancel

• In cmdStop, add this line of code: Bot.Stop

• In cmdCancel, add this line: Unload Me

There, now we have the form set up. Make sure Option Explicit is not at the top of the code for the form, then double-click on it and add the following code to the Form_Load procedure:

For Each AnimationName In Bot.AnimationNames
    lstAni.AddItem AnimationName
Next

This will loop through all the animations and add them to our list. You will see this soon. Add this to cmdPlay's code:

    On Error GoTo ErrOut
    Bot.Stop
    Bot.Play lstAni.Text
    Exit Sub

ErrOut:
    Bot.Speak "ERROR - UNABLE TO PLAY"

This will play the currently selected animation in the list. If there is an error, our bot will tell us.

Now, we need to add the code for 'Auto Animation' when the user says 'Animate'. It will choose one from the list, play it, and never show the form. Add a public Boolean variable named SelfAni in the General Declarations section of the form, and add the following code below the other in the Form_Load procedure:

If SelfAni Then
    Dim r As Integer
    r = Int(lstAni.ListCount * Rnd)
    lstAni.ListIndex = r
    Bot.Stop
    Bot.Play lstAni.Text
    Unload Me

End If

If SelfAni is true, then it will not display the form and choose a random animation. You will see how we will do this momentarily, so add the following code in the Phrase_Finish procedure:

Case "Display Animation List"

    frmAni.SelfAni = False
    frmAni.Show

Case "Animate"
    frmAni.SelfAni = True
    frmAni.Show

Run the program and test out our two new commands. What do you think?

System Information

Another very easy topic. Enter the following code into the Phrase_Finish procedure:

Case "Display System Information"
    Shell "C:\Program Files\Common Files\Microsoft Shared\MSINFO\msinfo32.exe", vbNormalFocus

This program will display your systems information. Our bot has to do something that has to do with our computer.

Shut Down

These are the two last commands. By now, my wrists are getting pretty sore from typing all of this! Add the following code above everything in modMain:

Declare Function ExitWindowsEx Lib "user32" (ByVal uFlags As Long, ByVal dwReserved As Long) As Boolean
Public Const EWX_LOGOFF = 0
Public Const EWX_REBOOT = 2
Public Const EWX_SHUTDOWN = 1

Now, we are almost done. As you know, we are only doing Reboot and Shutdown. You can program Logoff, which is pathetically simple (as you will soon see). Add the following code to finish this off:

Case "System Reboot"
    Dim reboot As Variant
    reboot = MsgBox("Are you sure you want to reboot?", vbYesNo, "Reboot?")

    If reboot = vbYes Then
        Screen.MousePointer = vbHourglass
        ExitWindowsEx EWX_REBOOT, 0
        Unload Me
        End
    End If

Case "System Shutdown"
    Dim down As Variant
    down = MsgBox("Are you sure you want to shutdown?", vbYesNo, "Shutdown?")

    If down = vbYes Then
        Screen.MousePointer = vbHourglass
        ExitWindowsEx EWX_SHUTDOWN, 0
        Unload Me
        End
    End If

Tips

Well, that is it! We are done. But before I go, here are some tips and notes:

• You can use Office's agents, and office can use the ones you downloaded

• I just scrapped the surface, there is much more to learn.

• Add another agent, you can use more than one!

• If you have any questions, E-mail them to me!

• Please Vote!!!